DATA ANNOTATION

Advancing African-Accented English Speech Recognition: Epistemic Uncertainty-Driven Data Selection for Generalizable ASR Models (2025.acl-srw)

Challenge:	Extreme multi-label classification (OXMC) is a challenging and critical task in natural language processing.
Approach:	They propose to use PUSL to reframe OXMC as an infinite keyphrase generation task . they propose to adopt evaluation metrics to reliably assess OXML models with incomplete ground truths.
Outcome:	The proposed approach improves on a highly imbalanced e-commerce dataset with missing labels . it generates 30% more unique labels and 72% of its predictions align with actual user queries .

Copied to clipboard

Bonaventure F. P. Dossou

Multimodality

Challenge:	Accents play a pivotal role in shaping human communication, a new study finds . existing ASR systems often perform inadequately, even mispronouncing African names .
Approach:	They propose a method that uses epistemic uncertainty to automate annotation to reduce costs and human labor.
Outcome:	The proposed method reduces costs and human labor by reducing data annotation and epistemic uncertainty.

Multi-News+: Cost-efficient Dataset Cleansing via LLM-based Data Annotation (2024.emnlp-main)

Copied to clipboard

Juhwan Choi, JungMin Yun, Kyohoon Jin, YoungBin Kim

Green & Sustainable NLP

Challenge:	Various attempts to correct noisy data in the construction process have been made, but human annotation is expensive and time-consuming.
Approach:	They propose to use large language models for data annotation to imitate human annotation and classify unrelated documents from a multi-document summarization task.
Outcome:	The proposed method imitates human annotation and classifies unrelated documents from the Multi-News dataset.

Identifying and Resolving Annotation Changes for Natural Language Understanding (2021.naacl-industry)

Copied to clipboard

Jose Garrido Ramas, Giorgio Pessot, Abdalghani Abujabal, Martin Rajman

Challenge:	Annotation conflict resolution is crucial for machine learning, says a new study . past work on annotation conflict resolution assumed data is collected at once . a a supervised neural model can resolve conflicts in data annotation but requires access to high-quality data .
Approach:	They propose an approach to resolve annotation conflicts in a real-world context using a German dialog system.
Outcome:	The proposed approach improves on a real-world dataset with 3.5M utterances in German.

GPTs Are Multilingual Annotators for Sequence Generation Tasks (2024.findings-eacl)

Copied to clipboard

Juhwan Choi, Eunju Lee, Kyohoon Jin, YoungBin Kim

Multilinguality

Challenge:	Existing methods of data annotation are time-consuming and expensive . complexity of crowdsourcing increases when dealing with low-resource languages .
Approach:	They propose an autonomous method to gather unlabeled data and label them using large language models.
Outcome:	The proposed method is cost-efficient and applicable for low-resource language annotation.

Mapping (Dis-)Information Flow about the MH17 Plane Crash (D19-50)

Copied to clipboard

Mareike Hartmann, Yevgeniy Golovchenko, Isabelle Augenstein

Information Retrieval

Challenge:	Digital media enables fast sharing of information, but also disinformation . studies on the spread of disinformation on social media focused on small, manually annotated datasets or used proxys for data annotation.
Approach:	They propose to use text classifiers to label Twitter content related to the MH17 crash to improve annotation accuracy.
Outcome:	The proposed classifier improves over a hashtag-based baseline, but still remains a challenge in labelling pro-Russian and pro-Ukrainian content with high precision.

EVIDENCEMINER: Textual Evidence Discovery for Life Sciences (2020.acl-demos)

Copied to clipboard

Xuan Wang, Yingjun Guan, Weili Liu, Aabhas Chauhan, Enyi Jiang, Qi Li, David Liem, Dibakar Sigdel, John Caufield, Peipei Ping, Jiawei Han

Information Retrieval

Challenge:	EVIDENCEMINER is a web-based system that allows users to query a natural language statement and retrieve textual evidence from a background corpora for life sciences.
Approach:	They propose a web-based system that lets users query a natural language statement and automatically retrieves textual evidence from a background corpora for life sciences.
Outcome:	EVIDENCEMINER is a web-based system that lets users query a natural language statement and automatically retrieves textual evidence from a background corpora for life sciences.

Self-Supervised Knowledge Triplet Learning for Zero-Shot Question Answering (2020.emnlp-main)

Copied to clipboard

Pratyay Banerjee, Chitta Baral

Two Contrasting Data Annotation Paradigms for Subjective NLP Tasks (2022.naacl-main)

Challenge:	Current supervised Question Answering methods rely on expensive data annotations and can introduce unintended annotator bias.
Approach:	They propose a self-supervised task over knowledge graphs that can be supervised by a data annotation tool.
Outcome:	The proposed task performs better than pre-trained language models on a large dataset.

Copied to clipboard

Paul Rottger, Bertie Vidgen, Dirk Hovy, Janet Pierrehumbert

FAMULUS: Interactive Annotation and Feedback Generation for Teaching Diagnostic Reasoning (D19-3)

Challenge:	Labelled data is the foundation of most natural language processing tasks, but there are valid beliefs about what the correct data labels should be.
Approach:	They propose two contrasting paradigms for data annotation that encourage annotator subjectivity . they propose a descriptive paradigm that allows for the surveying and modelling of different beliefs .
Outcome:	The proposed paradigms encourage annotator subjectivity, while the prescriptive paradigm discourages it.

Copied to clipboard

Jonas Pfeiffer, Christian M. Meyer, Claudia Schulz, Jan Kiesewetter, Jan Zottmann, Michael Sailer, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych

A Thesis Proposal ClaimInspector Framework: A Hybrid Approach to Data Annotation using Fact-Checked Claims and LLMs (2024.eacl-srw)

Challenge:	Existing systems for technologyenhanced learning address skills on recalling, explaining, and applying knowledge, e.g., in automatically generated language learning exercises and math word problems.
Approach:	They propose to leverage a NLP model to support experts in their further data annotation with automatic suggestions and provide automatic feedback for students.
Outcome:	The proposed system improves on two user studies on diagnostic reasoning in medicine and teacher education and can be extended to further use cases.

Copied to clipboard

Basak Bozkurt

Few-Shot Event Argument Extraction Based on a Meta-Learning Approach (2024.naacl-srw)

Challenge:	Current efforts focus on textual claims sourced mainly from Twitter . lack of automated control measures and reliance on human annotation increase noise risk .
Approach:	They propose to use a framework to integrate data annotation to mitigate misinformation . they propose to include fact-checks alongside the corresponding claims made by politicians .
Outcome:	The proposed dataset will include fact-checks alongside the corresponding claims made by politicians.

Copied to clipboard

Aboubacar Tuo, Romaric Besançon, Olivier Ferret, Julien Tourille

Selective Annotation via Data Allocation: These Data Should Be Triaged to Experts for Annotation Rather Than the Model (2024.findings-emnlp)

Challenge:	Recent studies on few-shot event extraction focus on event trigger detection and argument extraction in meta-learning contexts.
Approach:	They propose to use prototypical networks to perform few-shot event argument extraction . they propose to inject syntactic knowledge into the model to enhance relation embeddings .
Outcome:	The proposed approach achieves strong performance on ACE 2005 in several few-shot configurations.

Copied to clipboard

Chen Huang, Yang Deng, Wenqiang Lei, Jiancheng Lv, Ido Dagan

Efficient Argument Structure Extraction with Transfer Learning and Active Learning (2022.findings-acl)

Challenge:	Existing methods to obtain high-quality annotations under limited budgets focus on selecting informative data for expert annotations while the rest of the data is assigned to model annotation.
Approach:	They propose a semi-automatic annotation framework that uses error-aware triage and bi-weighting mechanisms to obtain high-quality annotations under limited budget.
Outcome:	The proposed framework outperforms baselines in the data annotation problem under limited budgets.

Copied to clipboard

Xinyu Hua, Lu Wang

Large Language Models for Data Annotation and Synthesis: A Survey (2024.emnlp-main)

Challenge:	Identifying and understanding the argumentative discourse structure in text has been a critical task in argument mining.
Approach:	They propose a context-aware Transformer-based argument structure prediction model that outperforms models that rely on features or only encode limited contexts.
Outcome:	The proposed model outperforms models that rely on features or encode limited contexts on five domains and on peer reviews on five different domains.

Copied to clipboard

Zhen Tan, Dawei Li, Song Wang, Alimohammad Beigi, Bohan Jiang, Amrita Bhattacharjee, Mansooreh Karami, Jundong Li, Lu Cheng, Huan Liu

A Practical Incremental Learning Framework For Sparse Entity Extraction (C18-1)

Challenge:	Existing surveys focus on LLMs' specific utility for data annotation and synthesis.
Approach:	They propose to use large language models to generate annotations from raw data . they also propose to review learning strategies for models utilizing LLM-generated annotations .
Outcome:	The proposed models can be used to improve the efficacy of machine learning models by generating and labeling raw data with relevant information.

Copied to clipboard

Hussein Al-Olimat, Steven Gustafson, Jason Mackay, Krishnaprasad Thirunarayan, Amit Sheth

Modelling Variability in Human Annotator Simulation (2024.findings-acl)

Challenge:	Existing approaches to extract entities from textual data are expensive and unattractive due to the high cost of training.
Approach:	They propose a framework that integrates Entity Set Expansion and Active Learning to reduce the cost of data annotation.
Outcome:	The proposed framework reduces the cost of sparse entity annotation by 85% and 45% while maintaining high accuracy.

Copied to clipboard

Wen Wu, Wenlin Chen, Chao Zhang, Phil Woodland

Green & Sustainable NLP

Challenge:	Human annotator simulation (HAS) is a cost-effective alternative to human evaluation tasks.
Approach:	They propose a framework to model human annotation variability via meta-learning . conditional softmax flow model leverages diverse human annotations via meta learning . results demonstrate that method can predict aggregated behaviours of human annotators .
Outcome:	The proposed method achieves state-of-the-art performance on two real-world human evaluation tasks: emotion recognition and toxic speech detection.

OmniTab: Pretraining with Natural and Synthetic Data for Few-shot Table-based Question Answering (2022.naacl-main)

Copied to clipboard

Zhengbao Jiang, Yi Mao, Pengcheng He, Graham Neubig, Weizhu Chen

Natural Language Interfaces

Challenge:	a table-based question answering system requires complex reasoning and alignment between questions and tables.
Approach:	They propose a table-based QA model that consumes both natural and synthetic data . they combine retrieval with masking to pair natural sentences with QA .
Outcome:	The proposed model outperforms existing models in few-shot and full settings and on WikiTableQuestions.

CrowdAgent: Multi-Agent Managed Multi-Source Annotation System (2025.emnlp-demos)

Copied to clipboard

Maosheng Qin, Renyu Zhu, Mingxuan Xia, null Chenchenkai, Zhen Zhu, Minmin Lin, Junbo Zhao, Lu Xu, Changjie Fan, Runze Wu, Haobo Wang

Citation-Enhanced Generation for LLM-based Chatbots (2024.acl-long)

Challenge:	Recent approaches to annotate data focus on labeling, but lack holistic process control . a novel system that integrates task assignment, data annotation, and quality/cost management is needed .
Approach:	They propose a multi-agent system that integrates task assignment, data annotation, and quality/cost management.
Outcome:	The proposed system automates human management by using a collaborative multi-agent system.

Copied to clipboard

Weitao Li, Junkai Li, Weizhi Ma, Yang Liu

Natural Language Interfaces

Challenge:	Existing efforts to alleviate hallucination in chatbots require additional training and data annotation.
Approach:	They propose a Citation-Enhanced Generation approach that uses retrieval argumentation to generate citations and a natural language inference-based citation generation module to generate content.
Outcome:	The proposed method outperforms state-of-the-art methods on three benchmarks.

Proposal: From One-Fit-All to Perspective Aware Modeling (2025.acl-srw)

Copied to clipboard

Leixin Zhang

Challenge:	Variation in human annotation and human perspectives has drawn increasing attention in natural language processing research.
Approach:	They propose to use annotation formats that better capture granularity and uncertainty of individual judgments and annotation modeling that leverages socio-demographic features to better represent and predict underrepresented or minority perspectives.
Outcome:	The proposed tasks aim to advance natural language processing research towards more faithfully reflecting the diversity of human interpretation, enhancing both inclusiveness and fairness in language technologies.

Controllable Mixed-Initiative Dialogue Generation through Prompting (2023.acl-short)

Copied to clipboard

Maximillian Chen, Xiao Yu, Weiyan Shi, Urvi Awasthi, Zhou Yu

Dialogue Systems & Conversational Agents

Challenge:	Mixed initiative dialogue systems allow all interacting agents to initiate actions to control the interaction.
Approach:	They propose to prompt large language models as a drop-in replacement for fine-tuning on conditional generation.
Outcome:	The proposed prompts improve fine-tuning and ground truth responses . the results show that generated responses are high .

AcTune: Uncertainty-Based Active Self-Training for Active Fine-Tuning of Pretrained Language Models (2022.naacl-main)

Copied to clipboard

Yue Yu, Lingkai Kong, Jieyu Zhang, Rongzhi Zhang, Chao Zhang

Self Question-answering: Aspect-based Sentiment Analysis by Role Flipped Machine Reading Comprehension (2021.findings-emnlp)

Challenge:	Existing methods for fine-tuning pre-trained language models ignore the potential of unlabeled data.
Approach:	They propose a framework that allows users to unleash the power of unlabeled data via self-training.
Outcome:	The proposed framework outperforms active learning and self-training baselines and improves the label efficiency of PLM fine-tuning by 56.2% on average.

Copied to clipboard

Guoxin Yu, Jiwei Li, Ling Luo, Yuxian Meng, Xiang Ao, Qing He

Reassessing Active Learning Adoption in Contemporary NLP: A Community Survey (2026.eacl-long)

Challenge:	unified Aspect-based Sentiment Analysis (ABSA) aims to couple aspect terms with their corresponding opinion terms, which might make it easier to predict sentiment polarities.
Approach:	They propose a new paradigm to pair aspect terms with their corresponding opinion terms . they propose to use a machine learning paradigm to solve the unified ABSA task .
Outcome:	The proposed framework can solve the ABSA task without any additional data annotation or transformation.

Copied to clipboard

Julia Romberg, Christopher Schröder, Julius Gonsior, Katrin Tomanek, Fredrik Olsson

Prompt Candidates, then Distill: A Teacher-Student Framework for LLM-driven Data Annotation (2025.acl-long)

Challenge:	a longstanding strategy to reduce annotation costs is active learning . data annotation is expected to remain important and active learning to stay relevant .
Approach:	They conduct an online survey to assess the perceived relevance of data annotation and active learning . they propose a strategy to reduce annotation costs using active learning, an iterative process .
Outcome:	The proposed strategies reduce setup complexity and uncertainty cost while maintaining model performance.

Copied to clipboard

Mingxuan Xia, Haobo Wang, Yixuan Li, Zewei Yu, Jindong Wang, Junbo Zhao, Runze Wu

Toward Fully Exploiting Heterogeneous Corpus:A Decoupled Named Entity Recognition Model with Two-stage Training (2021.findings-acl)

Challenge:	Existing methods for data annotation use an aggressive approach prompting LLMs to determine a single gold label for each unlabeled sample.
Approach:	They propose a teacher-student framework that distills candidate annotations with a Small Language Model (SLM) they propose to use LLMs to generate and distill candidate annotation with slms to ensure unique labels are provided for downstream tasks.
Outcome:	The proposed method outperforms existing methods due to uncertainty in LLMs and is noisetolerant.

Copied to clipboard

Yun Hu, Yeshuang Zhu, Jinchao Zhang, Changwen Zheng, Jie Zhou

Sensitivity, Performance, Robustness: Deconstructing the Effect of Sociodemographic Prompting (2024.eacl-long)

Challenge:	Named Entity Recognition (NER) is a fundamental and widely used task in natural language processing.
Approach:	They propose a decoupled NER model with two-stage training to take advantage of heterogeneous corpus, including dictionaries, distantly supervised instances, and human-annotated instances.
Outcome:	Empirical results show that the proposed model improves against baselines and can be scaled to a large extent.

Copied to clipboard

Tilman Beck, Hendrik Schuff, Anne Lauscher, Iryna Gurevych

From Multiple-Choice to Extractive QA: A Case Study for English and Arabic (2025.coling-main)

Challenge:	Existing studies on sociodemographic prompting have not explored the effectiveness of this technique.
Approach:	They propose to use sociodemographic prompting to steer models towards answers that humans with specific sociodemography would give.
Outcome:	The proposed technique can improve zero-shot learning by focusing on human sociodemographic profiles.

Copied to clipboard

Teresa Lynn, Malik H. Altakrori, Samar M. Magdy, Rocktim Jyoti Das, Chenyang Lyu, Mohamed Nasr, Younes Samih, Kirill Chirkunov, Alham Fikri Aji, Preslav Nakov, Shantanu Godbole, Salim Roukos, Radu Florian, Nizar Habash

Question Answering

Challenge:	Recent years have brought about very fast developments in Natural Language Processing (NLP), but many other languages are overlooked due to limited resources.
Approach:	They propose to repurpose a multilingual BELEBELE dataset for a task of extractive QA in the style of machine reading comprehension.
Outcome:	The proposed approach could be used to extract QA in the style of machine reading comprehension.

WASA: A Web Application for Sequence Annotation (L18-1)

Copied to clipboard

Fahad AlGhamdi, Mona Diab

Cross-Lingual Question Answering over Knowledge Base as Reading Comprehension (2023.findings-eacl)

Challenge:	a major barrier to research on CS has been the lack of large multilingual, multi-genre CS-annotated corpora.
Approach:	They propose a web-based annotation system that manages large-scale CS data annotation.
Outcome:	The proposed system can manage large-scale multilingual code switching (CS) data annotation.

Copied to clipboard

Chen Zhang, Yuxuan Lai, Yansong Feng, Xingyu Shen, Haowei Du, Dongyan Zhao

An End-to-End Submodular Framework for Data-Efficient In-Context Learning (2024.findings-naacl)

Challenge:	Existing high-quality xMRC datasets can be further utilized to fine-tune our model.
Approach:	They propose a cross-lingual question answering over knowledge base approach that converts KB subgraphs into passages to narrow the gap between KB schemas and questions.
Outcome:	The proposed approach outperforms baselines and achieves strong few-shot and zero-shot performance on two xKBQA datasets in 12 languages.

Copied to clipboard

Lilly Kumari, Shengjie Wang, Arnav Das, Tianyi Zhou, Jeff Bilmes

Visual Supervision in Bootstrapped Information Extraction (D18-1)

Challenge:	Recent advances in natural language tasks leverage the emergent In-Context Learning ability of pretrained Large Language Models (LLMs).
Approach:	They propose a framework for exemplar selection for in-context learning that uses a pool-based active learning approach to select Diverse and informative exemplars from the target tasks’ unlabeled pool.
Outcome:	The proposed framework outperforms existing methods for data annotation and similarity-based methods for test query-specific exemplar retrieval on 7 different NLP datasets and 5 LLMs of varying complexities.

Copied to clipboard

Matthew Berger, Ajay Nagesh, Joshua Levine, Mihai Surdeanu, Helen Zhang

Annotating Online Misogyny (2021.acl-long)

Challenge:	a list-based interface populated with informative samples is effective for data annotation . a 2D scatterplot populated by diverse and representative samples yields improved models .
Approach:	They propose a list-based interface that can be used to build efficient and effective data annotation models.
Outcome:	The proposed model learns the distributional similarity of entities through the patterns that match them in a large corpus while being discriminative with respect to human-labeled and machine-promoted entities.

Copied to clipboard

Philine Zeinert, Nanna Inie, Leon Derczynski

Investigating the Emergent Audio Classification Ability of ASR Foundation Models (2024.naacl-long)

Challenge:	Online misogyny is a category of online abusive language with serious and harmful social consequences.
Approach:	They propose an iterative annotation process and a taxonomy of labels for annotating misogyny in natural written language and cite a high-quality dataset of annotated posts from social media posts.
Outcome:	The proposed method aims to identify misogynistic language in natural written language and annotate it in social media posts using a high-quality dataset.

Copied to clipboard

Rao Ma, Adian Liusie, Mark Gales, Kate Knill

Speech & Audio in NLP

Challenge:	Text and vision foundation models can perform many tasks in a zero-shot setting . however, there has been little work on the zero-shoot abilities of ASR foundation models .
Approach:	They investigate the ability of ASR foundation models to perform zero-shot audio classification using text prompts and a decoding probability generator.
Outcome:	The proposed model outperforms state-of-the-art models on audio classification datasets without training them on extra data or adding any parameters.

LESA: Linguistic Encapsulation and Semantic Amalgamation Based Generalised Claim Detection from Online Content (2021.eacl-main)

Copied to clipboard

Shreya Gupta, Parantak Singh, Megha Sundriyal, Md. Shad Akhtar, Tanmoy Chakraborty

Multi-Task Pre-Training for Plug-and-Play Task-Oriented Dialogue System (2022.acl-long)

Challenge:	Existing work on claim detection is built on the basis of a 'segregation' of claims across different domains.
Approach:	They propose a generalized generalized model that captures syntactic features through part-of-speech and dependency embeddings, as well as contextual features through a fine-tuned language model.
Outcome:	The proposed model outperforms baselines on six claim datasets by an average of 3 claim-F1 points and 2 claim-f1 points on the general-domain experiments.

Copied to clipboard

Yixuan Su, Lei Shu, Elman Mansimov, Arshit Gupta, Deng Cai, Yi-An Lai, Yi Zhang

Self-Foveate: Enhancing Diversity and Difficulty of Synthesized Instructions from Unsupervised Text via Multi-Level Foveation (2025.findings-acl)

Challenge:	Existing pre-trained language models often form a cascaded generation problem . this can lead to error accumulation across different sub-tasks and greater data annotation overhead.
Approach:	They propose a plug-and-play model for task-oriented dialogue that learns primary TOD task completion skills from heterogeneous dialog corpora.
Outcome:	The proposed model learns primary TOD task completion skills from heterogeneous dialog corpora.

Copied to clipboard

Mingzhe Li, Xin Lu, Yanyan Zhao

Grammatical Error Correction via Mixed-Grained Weighted Training (2023.findings-emnlp)

Challenge:	Existing methods for training large language models rely on human effort for data annotation.
Approach:	They propose an unsupervised method that generates unsupervised instruction from unsupervised text using a "Micro-Scatter-Macro" method that excavates fine-grained information embedded in unsupervised texts.
Outcome:	The proposed method improves diversity and difficulty of synthesized instructions across multiple unsupervised corpora and diverse model architectures.

Copied to clipboard

Jiahao Li, Quan Wang, Chiwei Zhu, Zhendong Mao, Yongdong Zhang

Syntactic Text Processing

Challenge:	Empirical evaluation shows that MainGEC achieves consistent and significant performance improvements on two benchmark datasets.
Approach:	They propose to use mixed-grained weighted training to improve the training effect for GEC by analyzing the inherent discrepancies in annotated training data.
Outcome:	Empirical results show that the proposed method achieves significant performance improvements on two benchmark datasets.

Pre-train or Annotate? Domain Adaptation with a Constrained Budget (2021.emnlp-main)

Copied to clipboard

Fan Bai, Alan Ritter, Wei Xu

Revisiting Sample Size Determination in Natural Language Understanding (2023.findings-acl)

Challenge:	Recent work shows that pre-training in-domain language models can boost performance when adapting to a new domain.
Approach:	They propose to combine annotation and pre-training to maximize performance under budget constraints.
Outcome:	The proposed approach is based on the annotation cost of three procedural text datasets and pre-training cost of 3 in-domain language models.

Copied to clipboard

Ernie Chang, Muhammad Hassan Rashid, Pin-Jie Lin, Changsheng Zhao, Vera Demberg, Yangyang Shi, Vikas Chandra

Modeling Human Subjectivity in LLMs Using Explicit and Implicit Human Factors in Personas (2024.findings-emnlp)

Challenge:	Recent work has sought to reduce the annotation costs through the use of active learning and data sampling.
Approach:	They propose to estimate the training sample size needed to achieve a targeted model performance based on small amount of training samples.
Outcome:	The proposed approach predicts model performance within a small margin of mean absolute error (0.9%) with only 10% data.

Copied to clipboard

Salvatore Giorgi, Tingting Liu, Ankit Aich, Kelsey Isman, Garrick Sherman, Zachary Fried, João Sedoc, Lyle Ungar, Brenda Curtis

Challenge:	Large language models (LLMs) are increasingly being used in human-centered social scientific tasks, such as data annotation, synthetic data creation, and engaging in dialog.
Approach:	They propose to prompt LLMs with human-like personas and ask them to answer as if they were a specific human, either explicitly, with exact demographics, political beliefs, and lived experiences, or implicitly via names prevalent in specific populations.
Outcome:	The proposed model is based on explicit, explicit, and implicit personas, and fails to show implicit biases.

Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference (2021.findings-emnlp)

Copied to clipboard

Canming Huang, Weinan He, Yongmei Liu

Learning to Detect Noisy Labels Using Model-Based Features (2022.findings-emnlp)

Challenge:	Recent methods based on pre-trained language models have shown strong supervised performance on commonsense reasoning.
Approach:	They propose to use a common framework to solve commonsense reasoning tasks using a dataset from NLI.
Outcome:	The proposed method achieves state-of-the-art unsupervised performance on two commonsense reasoning tasks.

Copied to clipboard

Zhihao Wang, Zongyu Lin, Junjie Wen, Xianxin Chen, Peiqi Liu, Guidong Zheng, Yujun Chen, Zhilin Yang

Neural Disambiguation of Lemma and Part of Speech in Morphologically Rich Languages (2020.lrec-1)

Challenge:	Existing approaches to reduce label noise rely on heuristics and sample losses.
Approach:	They propose a method that transfers the noise distribution to a clean set and trains a model to distinguish noisy labels from clean ones using model-based features.
Outcome:	Empirically, the proposed approach improves over strong baselines on a wide range of tasks including text classification and speech recognition.

Copied to clipboard

José María Hoya Quecedo, Koppatz Maximilian, Roman Yangarber

Syntactic Text Processing

Challenge:	a method for disambiguating the lemma and part of speech of ambiguous words is proposed . a morphological analyser produces multiple analyses for ambiguously words .
Approach:	They propose a method for disambiguating the lemma and part of speech of ambiguous words in context . they use a large un-annotated corpus of text and a morphological analyser to train neural networks on the output of the analyser .
Outcome:	The proposed method outperforms the state-of-the-art on POS and lemma disambiguation in morphologically rich languages using no manual disambiguations or data annotations.

Assessing the Quality of an Italian Crowdsourced Idiom Corpus:the Dodiom Experiment (2022.lrec-1)

Copied to clipboard

Giuseppina Morza, Raffaele Manna, Johanna Monti

Challenge:	a crowdsourcing experiment has been used to collect idiom-related language resources . the data were collected through a game-with-a-purpose .
Approach:	They propose to use a game-with-a-purpose to collect idiom-related language resources . they use criteria adopted for the data annotation and evaluation process .
Outcome:	The proposed project evaluated idiom-related language resources from a game-with-a-purpose . the results and future work are presented.

Eeny, meeny, miny, moe. How to choose data for morphological inflection. (2022.emnlp-main)

Copied to clipboard

Saliha Muradoglu, Mans Hulden

Morphology

Challenge:	Data scarcity is a major bottleneck for many natural language processing tasks . active learning aims to reduce the cost of data annotation by selecting the most informative examples to label.
Approach:	They propose to use oracle experiments to select data that is most informative for the model.
Outcome:	The proposed sampling strategies show that they improve on the oracle experiment and the 10-cycle iteration using Natügu as a case study.

Conflicts in Texts: Data, Implications and Challenges (2025.findings-emnlp)

Copied to clipboard

Siyi Liu, Dan Roth

The Lessons of Developing Process Reward Models in Mathematical Reasoning (2025.findings-acl)

Challenge:	Conflicts in data could reflect complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs.
Approach:	This survey categorizes conflicting information into three key areas . they identify the areas where conflicting data can be ignored and undermine models' reliability and trustworthiness.
Outcome:	The findings highlight key challenges and future directions for developing conflict-aware NLP systems that can reason over and reconcile conflicting information more effectively.

Copied to clipboard

Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, Junyang Lin

infoVerse: A Universal Framework for Dataset Characterization with Multidimensional Meta-information (2023.acl-long)

Challenge:	a recent study shows that process reward models can make mistakes, leading to wrong conclusions.
Approach:	They propose a consensus filtering mechanism that integrates MC estimation with LLM-as-a-judge to improve model performance and data efficiency.
Outcome:	The proposed model outperforms existing open-source alternatives and provides practical guidelines for future research.

Copied to clipboard

Jaehyung Kim, Yekyung Kim, Karin de Langis, Jinwoo Shin, Dongyeop Kang

An Imitation Game for Learning Semantic Parsers from User Interaction (2020.emnlp-main)

Challenge:	Several methods for characterizing datasets based on model-driven meta-information have been developed, but the relationship and complementary effects of these methods have received less attention.
Approach:	They propose a framework that captures multidimensional characteristics of datasets by incorporating various model-driven meta-information.
Outcome:	The proposed framework outperforms baselines in three real-world applications and can be used in a variety of real-time problems.

Copied to clipboard

Ziyu Yao, Yiqi Tang, Wen-tau Yih, Huan Sun, Yu Su

Challenge:	Existing methods for learning semantic parsers are expensive and tedious . despite the widespread applications, bootstrapping and fine-tuning is tedious a task .
Approach:	They propose an alternative method for learning semantic parsers directly from users . they propose an annotation-efficient imitation learning algorithm that iteratively collects new datasets .
Outcome:	The proposed method is cost-effective and shows promising performance on the text-to-SQL problem.

Classifying Implant-Bearing Patients via their Medical Histories: a Pre-Study on Swedish EMRs with Semi-Supervised GanBERT (2022.lrec-1)

Copied to clipboard

Benjamin Danielsson, Marina Santini, Peter Lundberg, Yosef Al-Abasse, Arne Jonsson, Emma Eneling, Magnus Stridsman

NuNER: Entity Recognition Encoder Pre-training via LLM-Annotated Data (2024.emnlp-main)

Challenge:	Identifying the presence of implants in certain patients is important for radiologists because some implants are not compatible with MRI scanning.
Approach:	They compare the performance of two BERT-based text classifiers whose task is to classify patients as having or not having implant(s) in their body.
Outcome:	The proposed classifiers outperform fully-supervised classifier models on annotated data.

Copied to clipboard

Sergei Bogdanov, Alexandre Constantin, Timothée Bernard, Benoit Crabbé, Etienne Bernard

Towards Optimal Evaluation Efficiency for Large Language Models (2025.emnlp-main)

Challenge:	Named Entity Recognition (NER) is a core component of natural language processing, present in a variety of applications such as medical coding, financial news analysis, or legal documents parsing.
Approach:	They propose to use Large Language Models (LLMs) to create NuNER, a compact language representation model specialized in the Named Entity Recognition task.
Outcome:	The proposed model outperforms similar-sized foundation models in the few-shot regime and is based on a human-annotated dataset.

Copied to clipboard

Guohong Li, Deyi Xiong

ReactIE: Enhancing Chemical Reaction Extraction with Weak Supervision (2023.findings-acl)

Challenge:	Large language models (LLMs) require large-scale benchmarks, which are costly in terms of time, computational resources, or API tokens.
Approach:	They propose an efficient evaluation framework that selects a question subset based on pre-tested results and uses semantic analysis to evaluate whether the subset preserves the original benchmark.
Outcome:	The proposed evaluation framework outperforms previous methods in reliability and score accuracy.

Copied to clipboard

Ming Zhong, Siru Ouyang, Minhao Jiang, Vivian Hu, Yizhu Jiao, Xuan Wang, Jiawei Han

Domain Adaptation for Sentiment Analysis Using Robust Internal Representations (2023.findings-emnlp)

Challenge:	Structured chemical reaction information is a vital tool for chemists engaged in laboratory work and advanced endeavors such as computer-aided drug design.
Approach:	They propose a method which utilizes frequent patterns within the text as linguistic cues to identify specific characteristics of chemical reactions.
Outcome:	The proposed model outperforms baselines and outperformed existing models.

Copied to clipboard

Mohammad Rostami, Digbalay Bose, Shrikanth Narayanan, Aram Galstyan

Rethinking Annotation: Can Language Learners Contribute? (2023.acl-long)

Challenge:	Cross-domain sentiment analysis methods reduce the domain gap by training generalizable classifiers for each domain . large interclass margins in source domain help to reduce the effect of "domain shift" in the target domain.
Approach:	They propose a domain adaptation method which induces large margins between data representations that belong to different classes in an embedding space.
Outcome:	The proposed method reduces the domain gap by training cross-domain generalizable classifiers . large interclass margins in the source domain help reduce the effect of "domain shift" the proposed method is available in the u.s.

Copied to clipboard

Haneul Yoo, Rifki Afina Putri, Changyoon Lee, Youngin Lee, So-Yeon Ahn, Dongyeop Kang, Alice Oh

Challenge:	Researchers have traditionally recruited native speakers to provide annotations for benchmark datasets, but there are languages for which recruiting native speakers is difficult.
Approach:	They recruit 36 language learners and provide two types of additional resources and perform mini-tests to measure their language proficiency.
Outcome:	The proposed method improves learners' language proficiency in terms of vocabulary and grammar.

Align then Summarize: Automatic Alignment Methods for Summarization Corpus Creation (2020.lrec-1)

Copied to clipboard

Paul Tardy, David Janiszek, Yannick Estève, Vincent Nguyen

Text Generation

Challenge:	Summarizing text is not a straightforward task.
Approach:	They propose to use automated transcriptions to generate reports from automatic transcriptions as a dataset for neural summarization.
Outcome:	The proposed model improves on publicmeetings corpus on a dataset of aligned public meetings.

Mitigating Data Scarcity in Semantic Parsing across Languages with the Multilingual Semantic Layer and its Dataset (2024.findings-acl)

Copied to clipboard

Abelardo Carlos Martinez Lorenzo, Pere-Lluís Huguet Cabot, Karim Ghonim, Lu Xu, Hee-Soo Choi, Alberte Fernández-Castro, Roberto Navigli

PARROT: Zero-Shot Narrative Reading Comprehension via Parallel Reading (2023.findings-emnlp)

Challenge:	Large Language Models (LLMs) have advanced significantly in understanding human text, but semantic representations remain crucial for various applications.
Approach:	They introduce a multilingual semantic layer which decouples from disambiguation and external inventories and simplifies the task.
Outcome:	The proposed model reduces performance gap between languages and annotators by enabling them to understand semantic relations between concepts in any language.

Copied to clipboard

Chao Zhao, Anvesh Vijjini, Snigdha Chaturvedi

AnyTOD: A Programmable Task-Oriented Dialog System (2023.emnlp-main)

Challenge:	Existing approaches to narrative comprehension require extensive annotation of data.
Approach:	They propose a zero-shot approach for narrative comprehension through parallel reading using two parallel narratives that tell the same story.
Outcome:	The proposed approach surpasses previous zero-shot approaches and comparable performance to fully supervised models.

Copied to clipboard

Jeffrey Zhao, Yuan Cao, Raghav Gupta, Harrison Lee, Abhinav Rastogi, Mingqiu Wang, Hagen Soltau, Izhak Shafran, Yonghui Wu

Rethinking Table Instruction Tuning (2025.findings-acl)

Challenge:	a neuro-symbolic approach allows zero-shot adaptation to unseen tasks and domains . a neural LM keeps track of events that occur during a conversation and a symbolic program implements dialog policy is executed to recommend actions.
Approach:	They propose an end-to-end, zero-shot task-oriented dialog system . it is designed to adapt to unseen tasks or domains without prior training .
Outcome:	The proposed system can be programmed to adapt to unseen tasks without training . it reduces data collection and training requirements for enabling new TOD 1 16189 tasks .

Copied to clipboard

Naihao Deng, Rada Mihalcea

Multimodality

Challenge:	Existing studies have overlooked the impact of hyperparameters on table understanding abilities . authors show that smaller learning rates and fewer training instances can enhance table understanding while preserving general capabilities.
Approach:	They propose a hyperparameter-based instruction-tuned model for table-related tasks that improves out-of-domain table understanding ability and general capabilities.
Outcome:	The proposed model outperforms existing models on table-related tasks while maintaining strong out-of-domain generalization and general capabilities.

PolQA: Polish Question Answering Dataset (2024.lrec-main)

Copied to clipboard

Piotr Rybak, Piotr Przybyła, Maciej Ogrodniczuk

Natural Language Interfaces

Challenge:	Recent proposed systems for open-domain question answering (OpenQA) require large amounts of training data to achieve state-of-the-art performance.
Approach:	They propose an efficient annotation strategy that increases passage retrieval accuracy@10 by 10.55 p.p. while reducing the annotation cost by 82%.
Outcome:	The proposed approach increases passage retrieval accuracy @10 by 10.55 p.p. while reducing the annotation cost by 82%.

PRINCIPLES: Synthetic Strategy Memory for Proactive Dialogue Agents (2025.findings-emnlp)

Copied to clipboard

Namyoung Kim, Kai Tzu-iunn Ong, Yeonjun Hwang, Minseok Kang, Iiseo Jihn, Gayoung Kim, Minju Kim, Jinyoung Yeo

Dialogue Systems & Conversational Agents

Challenge:	Existing strategies for proactive dialogue face limitations such as limited strategy coverage and preference bias in planning.
Approach:	They propose a synthetic strategy memory for proactive dialogue agents based on large language models . PRINCIPLES is derived through offline self-play simulations and serves as reusable knowledge that guides strategy planning during inference.
Outcome:	PRINCIPLES is a synthetic strategy memory for proactive dialogue agents.

GUIDEX: Guided Synthetic Data Generation for Zero-Shot Information Extraction (2025.findings-acl)

Copied to clipboard

Neil De La Fuente, Oscar Sainz, Iker García-Ferrero, Eneko Agirre

Low-Resource NLP

Challenge:	Existing domain-specific IE systems require expert schema design, data annotation, and model training.
Approach:	They propose a method that automatically defines domain-specific schemas and infers guidelines and generates synthetically labeled instances.
Outcome:	The proposed method improves on seven zeroshot Named Entity Recognition benchmarks.

Balancing the Budget: Understanding Trade-offs Between Supervised and Preference-Based Finetuning (2025.acl-long)

Copied to clipboard

Mohit Raghavendra, Junmo Kang, Alan Ritter

Beyond Monolithic Rewards: Hybrid Multi-Aspect Reward Optimization (2026.findings-acl)

Challenge:	Results show that supervised fine-tuning and preference finetunation are the most efficient approaches for large language models.
Approach:	They propose to use Supervised Finetuning and Preference Finetunes to optimize training data budgets for Large Language Models.
Outcome:	The proposed approach improves performance on math tasks by 15% on the most expensive model, 1,000 examples.

Copied to clipboard

Radha Gulhane, Sathish Reddy Indurthi

Towards Autonomous Tool Utilization in Language Models: A Unified, Efficient and Scalable Framework (2024.lrec-main)

Challenge:	Existing approaches to optimize for multimodal learning use a single reward mechanism, but they lack confidence calibration across domains.
Approach:	They propose a hybrid reward and multi-aspect reward modeling framework that integrates model-based and rule-based reward paradigms for accuracy and confidence calibration.
Outcome:	The proposed model improves accuracy and confidence calibration across multimodal tasks and introduces a generalized length-penalty reward to stabilize training and improve performance.

Copied to clipboard

Zhi Li, Yicheng Li, Hequan Ye, Yin Zhang